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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS. 
WHICHEVER IS LONGER. FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1)S Responsive to comn:iunication(s) filed on 31 August 2006 . 
2a)^ This action is FINAL. 2b)n This action is non-final. 

3) G Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed In accordance with the practice under Ex parfe Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) 13 Claim(s) 1-4,6-10, 12-21.24,26,27. 34-37. 39'43A5-53.56 and 58 is/are pending in the application. 

4a) Of the above claim(s) Is/are withdrawn from consideration. 

5) n Claim(s) Is/are allowed. 

6) 13 Claim(s) 1-4. 6-10. 12-21. 24. 26. 27. 34-37. 39-43. 45-53. 56 and 58 is/are rejected. 
?)□ Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) 0 The specification is objected to by the Examiner. 

10)0 The drawing(s) filed on is/are: a)n accepted or b)n objected to by the Examiner. 

Applicant may not request that any objection to the drawing{s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)n Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)n All b)n Some * c)^ None of: 

1 .□ Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) [H Notice of References Cited (PTO-892) 4) Q Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) Paper No(s)/Mall Date. . 

3) □ Information Disclosure Statement{s) (PTO/SB/08) 5) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 

U.S. Patent and Tradenrtark Office 

PTOL-326 (Rev. 08-06) Office Action Summary Part of Paper No./Mail Date 20061 1 1 3 



Application/Control Number: 10/644,900 
Art Unit: 2176 



Page 2 



DETAILED ACTION 

1 ) This is a Final rejection in response to amendments/remarks filed on 9/21/2006. 

2) Claims 1 -4, 6-1 0, 1 2-21 , 24, 26, 27, 34-37, 39-43, 45-53, 56 and 58 are rejected. 
Claims 28-33 and 59-65 were previously withdrawn. Claims 5, 1 1, 22-23, 38, 44, 54, 55 
and 57 are cancelled. 

3) Effective filing date is 8/21/2003. 

4) Assignee is Microsoft. 

5) Examiner withdraws rejection under 35 (JSC 1 01 . 

Claim Rejections • 35 USC § 102 

6) The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless -(e) the invention was described in (1) an application for 
patent, published under section 122(b), by another filed in the United States before the invention by 
the applicant for patent or (2) a patent granted on an application for patent by another filed in the 
United States before the invention by the applicant for patent, except that an international application 
filed under the treaty defined in section 351(a) shall have the effects for purposes of this subsection of 
an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

6-1) Claims 1-4, 6-10, 12-21, 24, 26, 27, 34-37, 39-43, 45-53, 56 and 58 are 
rejected under 35 U.S.C. 102(e) as being anticipated by Schilit et al (US 6687876, 
filed Dec 30,1998). 

Regarding independent claims 1 and 34, Schilit teaches creating a first context node 
associated witli a first portion of a base portion of an electronic document; creating a 
second context node associated witti an annotation to ttie base portion, wlierein ttie first 
context node and ttie second context node are arranged in a single hierarchical data 
structure representing data associated with the electronic document; and linking the 
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second context node with the first context node. Schilit discloses a method for 
maintaining freeform ink annotations on changing views (Title), where layout 
relationships between the annotations and the associated objects are maintained (col 2, 
lines 57- 65) where a user using a hyperbolic browser makes a digital ink annotation on 
a node of a tree (col 3, lines 3-6; col 4, lines 45-54)(compare with the claimed first 
context node). Additionally, the relationship is between the ink annotations and objects 
in a view in the document (col 4, lines 45-54)(compare with claimed second context 
node). The system maintains the logical relationship between the freeform digital 
annotation and the object (col 4, lines 50-54)(compare with claimed the claimed linking) 
and maps the annotation to the text objects that they are on (col 7, lines 55-60). 
Regarding the hierarchical amendment, Schilit discloses mapping that may be to 
hierarchical object structures such as characters, phrases and paragraphs with mapping 
of annotations and characters or positions between characters (col 8, lines 29-36), 
where the hierarchical object structure can be applied to electronic book that comprises 
chapters, paragraphs, sentences, phrases, words, etc. (col 15, lines 23-40). The 
electronic book implies the book has subparts (ie., chapters, pages, ...) where the book 
and it's subparts form a single hierarchical structure. 

Regarding claims 2 and 35, Schilit teaches the first context node a word node. Schilit 
discloses annotated word (col 7, lines 30-32). 

Regarding claims 3, 20, 36 and 52, Schilit teaches wherein the first context node an 
ink drawing node. Schilit discloses annotations that are circles, margin notes, 
underlines/cross-outs (col 7, lines 50-57). 
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Regarding claims 4 and 37, Schilit teaches second context node is a group node. 
Schilit discloses grouping ink strokes on text to form a single freeform digital ink 
annotation (col 7, lines 48-51). 

Regarding claims 6 and 39, Schilit teaches the base portion is electronic text. Schilit 
discloses underlying text objects (col 4, lirie 67-col 5, line 1) and object are the text 
characters (col 5, lines 30-34; Fig 3A; 3B). 

Regarding claims 7 and 40, parsing at least the first portion of the base portion to 
thereby identify information for inclusion in the first context node. The Examiner 
interprets this claim limitation to mean parsing performed as a classification process to 
identify a stroke type, specifically to determine whether an electronic ink stroke is part of 
a drawing or handwritten text, in order to group ink strokes into meaningful associations 
such as lines, words and paragraphs (see specification, page 2, paragraph 4). Schilit 
discloses a control routine that groups ink strokes into various categories (col 10, lines 
1 5-20), by acquiring the region in the view that corresponds to the current character (col 
10, lines 65-67) and acquires a position corresponding to the current character position 
prior to setting the character position (col 11, lines 30-33), which the Examiner interprets 
as the same functionality as the claimed limitation. 

Regarding claims 8 and 41, Schilit teaches parsing at least the annotation to thereby 
identify information for inclusion in the second context node. Schilit discloses, by 
acquiring the region in the view that corresponds to the current character (col 10, lines 
65-67) and acquires a position corresponding to the current character position prior to 
setting the character position (col 11, lines 30-33), which the Examiner interprets as the 



Application/Control Number: 10/644,900 Page 5 

Art Unit: 2176 

same functionality as the claimed limitation. 

Regarding claims 9 and 42, Schiiit teaches prior to parsing the annotation, the 
annotation includes at least one unclassified ink node. Schiiit discloses unanchored 
segments of the freeform digital ink that will stretch to fit between the anchored points 
(col 6, lines 15-17), where the process first receives freeform digital ink annotations 
prior to anchoring them to the view (col 6, lines 46-49). 

Regarding claims 10, 24 and 43, Schiiit teaches rendering the base portion and the 
annotation, wherein the annotation is located at a first position with respect to the base 
portion; changing data associated with the base portion such that a location associated 
with the first context node changes to a second position; and rendering the annotation 
and the base portion with the changed data, wherein the annotation is rendered at a 
third position with respect to the base portion at least in part based on the second 
position of the first context node. 

First, The Examiner interprets this claim to mean changing the location of a base 
portion of the document that has attached ink annotations to another location and 
maintaining the logical connection of the ink annotation to the changed base portion of 
the changed portion in it's new location. 

Schiiit discloses a method that moves the anchor points if the underlying 
anchored object moves (col 6, lines 13-14) where the document view is updated when 
the document changes and the maintained mapping of the ink annotation is used to 
render the ink annotation on the changed object view (col 6, lines 60-63). The logical 
relationship between freeform digital ink annotation and the objects is maintained (col 3, 
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line 10-13) and the objects can be copied or moved along with the logical relationship 
with the annotation (col 3, lines 17-34). 

Regarding claims 12 and 45, Schilit teaches the first context node and the second 
context node share at least one common parent node. Schilit discloses that the 
annotation flows along with the underlying text objects in a logical relationship (col 4, 
line 66 - col 5, line 3). The Examiner interprets the annotation and text object are in a 
parent child relationship because the text object is under the annotation. 
Regarding claims 13 and 46, Schilit teaches data associated with the first context 
node and the second context node enable the electronic document to be rendered such 
that the annotation contains the first portion of the base document. Schilit teaches a 
circle annotation that is anchored to the span of words that is encircled by the circle (col 
5, lines 49-52; Fig 3A, item 68). 

Regarding claims 14 and 47, Schilit teaches data associated with the first context 
node and the second context node enable the electronic document to be rendered such 
that the annotation underlines the first portion of the base document. Schilit discloses 
underline annotations underneath the text and are mapped to the text objects that they 
are underneath 9col 7, lines 55-60). 

Regarding claims 15 and 48, Schilit teaches data associated with the first context 
node and the second context node enable the electronic document to be rendered such 
that the annotation strikes out the first portion of the base document. Schilit discloses 
cross-out annotations directly on the text and are mapped to the text objects that they 
are on (col 7, lines 55-60). 
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Regarding claims 16 and 49, Schilit teaches data associated with the first context 
node and the second context node enable the electronic document to be rendered such 
that a first portion of the annotation points between a second portion of the annotation 
and the first portion of the base document. Schilit teaches (see Fig 3A) a document 
having digital ink annotations (item 56) that includes a circle connected to an arrow 
where the tip of the arrow is anchored to it's end point while the line connecting the 
circle and the tip of the arrow is not anchored (col 5, lines 48-53). The Examiner 
interprets that the tip of the arrow points to another portion of the document that is 
between the annotation (the circle 68) and the base document (item 66). 
Regarding claims 17 and 27, Schilit teaches a computer-executable instructions 
stored thereon for performing the method of claim 1 , Schilit discloses a programmed 
general purpose computer to implement the system (col 4, lines 18-28). 
Regarding Independent claims 18 and 50, Schilit teaches first data set containing 
data associated with a base document; a second data set that includes unclassified 
electronic ink data; storing results from parsing the second data set as a data structure 
on a computer readable medium, wherein the first context node and the second context 
node are arranged in a single hierarchical data structure representing data associated 
with the electronic document and linking at least some portion of the second data set 
with at least some portion of the first data set. Schilit discloses a method for 
maintaining freeform ink annotations on changing views (Title), where layout 
relationships between the annotations and the associated objects are maintained (col 2, 
lines 57- 65) where a user using a hyperbolic browser makes a digital ink annotation on 
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a node of a tree (col 3, lines 3-6; col 4, lines 45-54)(compare with the claimed first 
context node). Additionally, the relationship is between the ink annotations and objects 
in a view in the document (col 4, lines 45-54)(compare with claimed second context 
node). The system maintains the logical relationship between the freeform digital 
annotation and the object (col 4, lines 50-54)(compare with claimed the claimed linking) 
and maps the annotation to the text objects that they are on (col 7, lines 55-60). 
Schilit's method invention is based upon the concept that anchors are determined for 
freeform digital ink annotations and the text editor stores the sequence and presentation 
of characters (col 8, lines 43-45). 

For the parsing portion of the limitation, The Examiner interprets this claim 
limitation to mean parsing performed as a classification process to identify a stroke type, 
specifically to determine whether an electronic ink stroke is part of a drawing or 
handwritten text, in order to group ink strokes into meaningful associations such as 
lines, words and paragraphs (see specification, page 2, paragraph 4). Schilit discloses 
a control routine that groups ink strokes into various categories (col 10, lines 15-20), by 
acquiring the region in the view that corresponds to the current character (col 10, lines 
65-67) and acquires a position corresponding to the current character position prior to 
setting the character position (col 1 1 , lines 30-33), which the Examiner interprets as the 
same functionality as the claimed limitation. 

Regarding claims 19 and 51, Schilit teaches the first context node a text word node. 
Schilit discloses annotated word (col 7, lines 30-32). 

Regarding claims 21 and 53, Schilit teaches during the linking, the first context node of 
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the first parsed data set is linked with the first context node of the second parsed data 
set. Schilit discloses a logical mapping relationship between the ink annotations and 
the text object of the document (col 4, lines 50-54; col 6, lines 45-50). 

Regarding claim 56, which claims substantially similar subject matter as claim 24 and 
is rejected under similar lines of rejection as the rejection of claim 24, above. 

Regarding claims 26 and 58, Schilit teaches at least one context node associated 
with the first data set shares at least one common parent node with at least one context 
node associated with the second data set. Schilit discloses that the annotation flows 
along with the underlying text objects in a logical relationship (col 4, line 66 - col 5, line 
3). The Examiner interprets the annotation and text object are in a parent child 
relationship because the text object is under the annotation. 

Response to Arguments 
Applicant's arguments filed 8/21/2006 have been fully considered but they are not 
persuasive. Regarding claim 1 and 34, Applicant argues that Schilit does not teach the 
amended limitations dealing with a single hierarchical data structure representing data 
associated with the electronic document (Remarks, page 16). The examiner disagrees. 
Specifically, Schilit discloses mapping that may be to hierarchical object structures such 
as characters, phrases and paragraphs with mapping of annotations and characters or 
positions between characters (col 6, lines 29-36), where the hierarchical object structure 
can be applied to electronic book that comprises chapters, paragraphs, sentences, 
phrases, words, etc. (col 15, lines 23-40). The electronic book implies the book has 
subparts (ie., chapters, pages, ...) where the book and it's subparts form a single 
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hierarchical structure. 

Regarding claiiT5l8 and 50, applicant argues that Shilit does not teach the amended 
limitation of storing from parsing two data sets as a data structure (Remarks, page 16, 
bottom - 17). The examiner disagrees. Specifically, Schilit's system maintains the 
logical relationship between the freeform digital annotation and the object (col 4, lines 
50-54)(compare with claimed the claimed linking) and maps the annotation to the text 
objects that they are on (col 7, lines 55-60). Schilifs method invention is based upon 
the concept that anchors are determined for freeform digital ink annotations and the text 
editor stores the sequence and presentation of characters (col 8, lines 43-45). 

Relevant Art Considered by not cited 
Nonpatent Literature "Robust Annotation Positioning in Digital Documents" by A.J. 
Bernheim Brush, David Bargeron, Anoop Gupta and JJ Cadiz (see IDS, 4/2006). 

Conclusion 

Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 
§ 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 
CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gautam Sain whose telephone number is 571-272- 
4096. The examiner can normally be reached on M-F 9-5 EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Heather Herndon can be reached on 571-272-4136. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status Information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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Supervisory Patent Examiner 
Technology Center 2 100 



